Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert to last know good docker build #668

Open
wants to merge 27 commits into
base: bkou_revert2
Choose a base branch
from
Open

Conversation

sfc-gh-bkou
Copy link

No description provided.

sfc-gh-afedorov and others added 27 commits May 14, 2024 09:20

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* Remove vestigial logging

no need to log because these errors are recorded into the "error" column in the corresponding table

* Add retry and update aio* dependencies

Update aiohttp, aioboto3, and aiobotocore dependencies and add retry logic to sts.assume_role invocation.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* Update base image to Debian bookworm

* Update to py38

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
)

This reverts commit a66a817.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
This reverts commit a80713c.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* Update Python and pip dependencies & fix tests

* fix share db view creation in installer script

Snowflake added a new column so the share db name is col4 now

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
…ory (#671)

* Tag image with commit SHA after build, and push to repository

* updates to contibution file

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
) (#673)

This reverts commit b7b5126.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
fixing error
> botocore.exceptions.CredentialRetrievalError: Error when retrieving credentials from container-role: Error retrieving metadata: Received error when attempting to retrieve container metadata: Connect timeout on endpoint URL: "http://<ip>/v2/credentials/<uuid>"

per https://repost.aws/questions/QUCFqv7OfoQlygJrmwfkJ24Q/various-aws-apis-fail-due-to-timeout
This reverts commit b84705b.
This reverts commit 6c0d2bd.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
…ed'` error (#675)

* Update azure_collect.py to fix `KeyError: 'managedByExtended'` error

* Update v1_9_6-v1_10_0.md
getting throttled at metadata service returning header "X-Rate-Limit-Limit: 40.00"

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
* fix awsic rate limiting and OOM errors

- rate limit metadata separately
- prune cache to fix oom errors
- remove / abstract out inline retries

* optimizations

- use boto's retry mechanism for 429 responses
- cache AssumeRole failures indefinitely
- re-use sessions between regions
- refactor API rate limits as AioRateLimit instances
- increase batch size to 1000

* add pause in pagination
Putting them last because they currently time out after a few days of ingestion
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants